arXiv:1503.05679vl [quant-ph] 19 Mar 2015 


Determination and correction of persistent biases in quantum annealers 

Alejandro Perdomo-Ortiz,^’Bryan O’Gorman,^’^ Joseph 
Fluegemann/A Rupak Biswas,^ and Vadim N. Smelyanskiy^ 

^Quantum Artificial Intelligence Lab., NASA Ames Research Center, Moffett Field, CA 94035, USA 
“^University of California Santa Cruz at NASA Ames Research Center, Moffett Field, CA 94035, USA 
^SCT Inc., 7701 Creenbelt Rd, Suite 400, Creenbelt, MD 20770, USA 
"^San Jose State Research Foundation at NASA Ames Research Center, Moffett Field, CA 94035, USA 
^Exploration Technology Directorate, NASA Ames Research Center, Moffett Field, CA 94035 
^ Coogle, 150 Main St, Venice Beach, CA, 90291 
(Dated: March 20, 2015) 

Calibration of quantum computing technologies is essential to the effective utilization of their 
quantum resources. Specifically, the performance of quantum annealers is likely to be significantly 
impaired by noise in their programmable parameters, effectively misspecification of the compu¬ 
tational problem to be solved, often resulting in spurious suboptimal solutions. We developed a 
strategy to determine and correct persistent, systematic biases between the actual values of the pro¬ 
grammable parameters and their user-specified values. We applied the recalibration strategy to two 
D-Wave Two quantum annealers, one at NASA Ames Research Center in Moffett Field, California, 
and another at D-Wave Systems in Burnaby, Canada. We show that the recalibration procedure 
not only reduces the magnitudes of the biases in the programmable parameters but also enhances 
the performance of the device on a set of random benchmark instances. 


I. INTRODUCTION 

Quantum annealing (QA) is a metaheuristic for solving 
combinatorial optimization problems [T]. The recent in¬ 
troduction of QA hardware by D-Wave Systems mg has 
invigorated theoretical and experimental research into 
the computational power and practical implementation 
challenges of the QA paradigm. Current research stud¬ 
ies focus on both fundamental and applied aspects, in¬ 
cluding application to real-world problems mg , crite¬ 
ria for detecting quantum speedup [9] , the computational 
role of quantum tunneling m , error-supression m, the 
relationship between classical simulated annealing and 
quantum annealing p!2HT8] . spin-glass perspectives on 
the hardness of computational problems uniEO], and pro¬ 
gramming strategies that address intrinsic noise [HIES]. 

The quantum annealers used for this study are of the 
second generation of D-Wave devices, also called D-Wave 
Two [2]: one located at NASA Ames Research Center in 
Moffett Field, California, (“NASA device”), and another 
located at D-Wave Systems in Burnaby, Canada (“Burn¬ 
aby device”). These consist of 64 unit cells of a previ¬ 
ously characterized eight-qubit unit cell [gug. In the 
NASA and Burnaby devices, post-fabrication character¬ 
ization determined that only 509 and 424 qubits, respec¬ 
tively, out of the 512 qubit arrays can be reliably used for 
computation. The array of coupled superconducting flux 
qubits is, effectively, an artificial Ising spin system with 
programmable spin-spin couplings and transverse mag¬ 
netic fields. It is designed to solve instances of the follow¬ 
ing (NP-hard [24]) classical optimization problem: Civen 
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a set of local fields {hi} C M and couplings {Jij} C M, 
find the assignment s* = • • • s*, that minimizes the 

objective function ^(s), 

A'Ising(s) — ^ ^ hiSi T ^ ^ JijSiSj, (1) 

l<i<n 

where \hi\ < 2, \Jij\ < 1, and Si G {±1}. Finding the 
optimal s* is equivalent to finding the ground state of 
the corresponding quantum Ising Hamiltonian iJjsing = 

1 where erf is Pauli ^ operator 
acting on the ith spin. More details of QA can be found 
in Appendix [A| 

Currently, D-Wave devices are only calibrated at the 
level of ensuring that the low-level control circuitry has 
its intended effect on the physical quantities like current, 
flux, etc., that it is meant to control [25] [26]. Early re¬ 
search into the performance of D-Wave devices has indi¬ 
cated the presence of significant imprecision in the set¬ 
ting of the fields that define the problem to be solved, 
a consequentially significant impairment to the success¬ 
ful solution of the problem ini UHlllOlIlT]. Recently, 
some work has used a phenomenological noise model of 
the fields [hi] and {Jij} in which the distributions of 
the deviations from the programmed values are given by 
Caussians with means zero and standard deviations, re¬ 
spectively, of 0.05 and 0.035 (in units of the maximal Jij), 
independently instantiated for each qubit and anneal and 
constant throughout the course of a given anneal. The 
parameters of the Caussians were derived by adding in 
quadrature the variances of several known microscopic 
sources of noise [26]. This model has been used in an 
attempt to explain the failure rate of D-Wave devices 
as partly due to misspecification of the programmable 
values. There are many sources of noise in quantum an¬ 
nealers, each with a different effect and time scale, and 
we address here only one manifestation. The variances 
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we cite for {hi} and {Jij} are in a sense incomparable 
to those just mentioned, and relevant only within the 
context of the experiments described below. 

The presence of systematic biases in quantum anneal¬ 
ers has been reported elsewhere [22] . The biases referred 
to there are fundamentally different in nature than the 
ones address here, in that the former are collective biases 
on the qubits of ferromagnetic chains that depend on the 
strength and topology of the couplings therein and are 
due to the noise specifically caused by those couplings, 
and they must be determined anew for each embedding 
topology used. In this work, we present a methodology 
for determining, in parallel, the persistent, systematic 
biases in all of the individually available programmable 
parameters of a quantum annealer that requires a rela¬ 
tively small amount of total annealing time. We show 
that correcting for these biases produces an increase in 
the quality of solutions found on a set of random bench¬ 
mark instances. The strategy presented here is the first 
proposal for a full-device recalibration at the user level, 
i.e. based only on the data from tailored instances and 
without access to the low-level control circuitry. 


II. DETERMINATION OF PERSISTENT 
BIASES 

Because actual quantum annealers operate at non-zero 
temperature, there exists some threshold for the values 
of the fields [hi] and couplings {Jij} below which ther¬ 
mal effects dominate the annealing process. When the 
strengths of the fields and couplings are set sufficiently 
small, the probabilities of the final states of the qubits 
are well described by a Boltzmann distribution. Roughly, 
the relevant energy scale is given by /cT, where k is Boltz¬ 
mann’s constant and T is the temperature. (Henceforth, 
we will work in units in which k = 1.) By running ex¬ 
periments in this regime, persistent biases in the values 
of the programmed fields can be uncovered. 


is the temperature of qubit i. This yields 
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(We assume that and Ti are constant at least over the 
course of the experiment.) More generally, we define the 
function a{p) = (l/2)ln[(l —p)/p]. Once experimental 

values of are obtained for various values of 

the data are fit to obtain the estimates of and T^. 


B. J biases 


The biases of the couplings between qubits can be 
determined in a similar fashion. Let Jij = J^j^ + J^j"^ 
be the effective value of the coupling between qubits 

i and j and Pij{ tt \h\^\hf \hP,hf\ jj;f, be 

the probability of qubits i and j both being in the 
spin-up state, where J^j^ is the programmed value of 
the coupler and J^j^ the bias, analogous to and hf\ 
respectively. Here, we set = 0, and write sim- 

ply Pij (tt I ) = Pij (tt I hf^, hP, j\f , pf) . Other 

probabilities pij (• - j and combinations thereof such 

as Pii (tt V tt \JP) ^ Pij (tt |4V Pv (tt \JP), 
are analogously denoted. 

One approach to determine the bias jj^j^ is to naively 

assume that = 0, in which case the thermal 

distribution is modeled by 


Pij{nvu\Jpj 
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A. h biases 

In our model, hi = + hf^ is the effective value 

of the local field of qubit i, where is the user- 
programmed value and hf^ is the -independent bias. 
In an ideal device, hf^ = 0. 

Let Pi (t I ^i^^) [Pi ( i I ^i^^) ] be the probability of 
qubit i being in the spin-up [spin-down] state at the end 

of an anneal with the programmed value A com¬ 

pletely thermal model for this probability is given by 

where ai{hP,hP,Ti) = hi/Ti = {h^ + hP)/Ti and Ti 


where aij{j\f) ^ Jij/Tij = {j\f + j\f)/Tij. For con- 
cision, we leave the dependence of pij and aij on hj, 

Jij\ and Tij implicit. Similarly to the case for hi, this 
yields 



1 -Pij(tt V U) 
Pij(tt V u) 


( 5 ) 


A more accurate estimate for J^^^ can be obtained 
by considering nonzero hP and hP (but still setting 
hP = hP = 0). Let 

Z= (.(sihf'>+s,hf^+s,sMf+4f))/Ti 

si,sje{±i} 


( 6 ) 






3 


be the partition function. Then 


Pij (TT ^Pij (-i-i) 


(7) 

(8) 
(9) 


or 




(p) 



Pij(U)Pij(-lt) 

Pij(tt)Pij(U)' 


( 10 ) 


Note that the assumption that = 0 implies 

that Piiitt) = Pij Hi) and Pijiti) = Pij (it), and that in 
this case ( |10| | reduces to (§. 


III. BIASES DETERMINED IN CURRENT 
D-WAVE DEVICES 

In this section and the next, we present preliminary re¬ 
sults from the application of the above methods to actual 
hardware, namely two D-Wave quantum annealers: the 
NASA and Burnaby devices. Data collection was severely 
constrained by repeated downtimes and subsequent recal¬ 
ibrations of the device at NASA, and the presently ongo¬ 
ing installation of a newer chip, as well as limited access 
to the device at D-Wave Systems. This manuscript will 
be updated once more complete data has been collected. 


A. h biases 

To experimentally determine the biases we set 

all = 0 and initially assume the effect of nonzero 

to be negligible. We therefore ran the exper¬ 
iments for all qubits within a given device simultane¬ 
ously, with the same value of for every 

working qubit. Each value of was run 100 times, 
where each run consisted of 1,000 annealing cycles. The 
probability ( t ) was calculated for each run r, and 
the median probability Pi( t taken over the 100 

{Pi^\ t calculated. From this, we define the 

“median’ll = a {pi 

Calculation of the biases Since ai = + jTi, 

for each qubit, we fit a line to d^)} by minimizing 

the quadratic loss; the resulting slope gives us an esti¬ 
mated inverse qubit temperature = 1/T^, and from 


^ Technically, this is a slight abuse of terminology; while the me¬ 
dian of {(^{Pi^^)} is almost the same as a(pi) because the func¬ 
tion a(p) is monotonic, the two quantities can differ slightly in 
the case of an even number of values. 


the intercept we can determine the bias in several 
ways. 

One way is to simply use the fitted parameters as is: 
= df^Ti. Experimental data, however, indicate that 
the estimates of the qubit “temperatures” calculated as 
above are not exactly that, but include in their calcu¬ 
lation effects other than that due to true variation in 
temperature between the qubits. Some estimate of a uni¬ 
form device temperature should therefore be used. (See 
App.j^for more detail.) In our experiments, we used two 
different quantities. The first is the “mean temperature”, 
TpW = 1 Ti, where Ti = 1/Pi and n is the number 
of (working) qubits. The second is the “median temper¬ 
ature” called thus not because it is the median of 

{Ti} but because it is calculated by taking the inverse 
of the slope of the line fit to the points 
where d^^^ (h^^^) is defined as the median over the qubits 
of {di{h^^^)}. In practice, the quantities and 
are effectively the same. 

Inherent noise limit Figures j^a,b,d,e) shows the 
probability = piF^'^^ { | \h[^^= O.l) for each of 
the qubits in two different unit cells and for each of the 
100 runs. From this we see two fundamental phenom- 
ena: the run-to-run fluctuation of the probability p\ , 
and the presence of some qubits for which the typical 
probability is well separated from the others, to a degree 
greater than the inherent variability. The latter indicates 
the presence of the qubit-specific biases that we hope to 
determine and correct, while the former provides a limit 
to the precision with which we can control the effective 
hi. We consider a calibration procedure successful if post 
hoc the qubits are unbiased to within this experiment 
limit. To compare the variability of {pi{h^^^)} over dif¬ 
ferent values of h^^\ for each qubit i and value of h^^^ 

we compute the 100 values of h\^"^ = a{p\^^ 
from the 100 runs. Figure [^e) shows the standard devi¬ 
ation cr~(^) of {^ 55 ^} over the 100 runs; the plot is typical. 

^55 

Importantly, the standard deviation cr~(^) is seemingly in- 

i 

dependent of the programmed for the values of h^'^^ 
considered. To characterize the typical variance of hi, for 
each qubit the mean of standard deviations {cr~(r)} 

was taken over the values of h^P\ Figure Bf) shows a 
histogram of this mean standard deviation over all of the 
qubits. The distribution is quite tight, with an average 
of =0.0156. This run-to-run variation in the estimate 

h 

of hi for a given h^^^ ultimately leads to a limit on the 
precision with which we can estimate hf^ , though quan¬ 
titatively the exact limit depends on the number of runs, 
number of reads per run, and values and number of h*^^^s 
examined. 

Figures 2[a,b) shows two windows of d^^^ (h^^^) for one 
experiment on the device at NASA Ames. Figure |^b) 
shows the range G [—0.1,0.!] used in the calculation 
of the biases. There, the linearity of d ^^^, and thus the ac¬ 
curacy of the thermal model, is evident; this is typical of 
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FIG. 1. Persistent systematic biases and inherent hi noise limit. (a,b,d,e) The probability (I = O.l) of qubit 
i being spin-down in the 100 runs {r} of 1000 reads each, for two experiments on the 16 qubits of two adjacent unit cells, done 
36 days apart and in which the programmed value of all the qubits was =0.1. Two important aspects are apparent that 
lay the foundation for the present work: there are qubits whose deviation from the rest is much greater than the noise level, and 
that this deviation persists for more than a month. As we show, these deviations in the probabilities are indicative of biases in 
the programmable parameters, (c) For a single, typical qubit, 55, the standard deviation over 100 runs of the estimates 
of /i 55 versus the programmed (f) The means {d= } over the programmed values of the standard deviations {cr= }, 

as in (c). The values of hi in (c,f) were calculated using the mean temperature 


all the experiments reported. Figure [^a) shows a wider 
window h^^^ G [—0.35,0.35], where the nonlinearity out¬ 
side of the former range is evident, indicating the failure 
of the thermal model for larger magnitudes of h ^^^, where 
annealing dynamics start to dominate thermal dynamics. 
Figure pic) shows revealing the limited reso¬ 

lution 04)25 of the digital-to-analog converters (DACs) 
used to implement /i 2 - Such stepping behavior is typical 
for all of the qubits. 


Narrowing of the bias distribution To show the cor- 
rectability of the persistent biases, we run the experiment 
described above repeatedly, each time attempting to cor¬ 
rect the biases using estimates thereof from the prior it¬ 
eration. Let hf'^^ be the experimentally deter¬ 

mined value of the bias h\^^ [median temperature 
based on the /^-iteration; by convention, set = 0. 

Let be the desired programmed value. In the k-th. 


iteration, we set 

k 

hiP) ^ ^ ^(P,0) _ ^ 

k' = l 

Figures [^d-e) show the distribution of = 

|^(6,/e)^(/i^/c) j second iterations, i.e. be¬ 

fore and after the recalibration procedure. The narrowing 
in the distribution is a clear indication that the procedure 
is working to remove the extreme biases. 

Further support for the success of recalibration is pro¬ 
vided by looking at the distribution over the qubits of 
the success probabilities for all values of h^P\ Figurej^f) 
shows a uniform reduction of the variance over the qubits 
in the values of h. The points are the mean h over the 
qubits of {hi^ for the corresponding h^'P\ and the shaded 
region indicates the standard deviaton. The narrowing 
of the distribution is clear evidence that the recalibration 
procedure not only narrows the distribution of the biases 
[Fig. [^d,e)], as reflected in the shift of the mean, but 
also reduces the variance for all values of h^^K 
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(a) (b) (c) 



h^p) h^p) h^p) 



FIG. 2. Detection and correction of systematic biases, (a) From a single experiment without correction, the median 
quantity over the qubits, of the quantities various values of in [-0.35,0.35]. (b) Same as (a), 

but only using values of in [-0.1,0.!]. Note the tightness of the fit to a line, indicating the validity of the thermal model, 
(c) From the same experiment as in (b), the quantity 02 for a single, typical qubit, 2. Note the step function resulting 

from the limited precision of the digital-to-analog converter used to control the held. (d,e) The estimated biases from 

two experiments k — 1,2. The hrst is without correction; the second was corrected using the biases estimated from the hrst. 
Note that the distribution signihcantly more narrowed and centered near zero after correction, (f) The average over the qubits 

of the quantites {hi} for different values of the programmed with error bands given by the standard deviation. As for 
(d,e), note the narrowing and centering of the distributions. 


B. J biases 

Calculation of the J biases In the data presented 
here, the J biases were determined using Eq. The 
programmable value of the local helds were uni¬ 

formly set to zero. For a given value of J^'^\ the exper¬ 
iment was run in six batches. In each batch, the pro¬ 
grammed coupling was uniformly set to for a 
each coupler of a pairwise disjoint subset of all of the 
couplers, and for each of the rest was set to zero. 
Over the six batches, each coupling was set to 
exactly once. As for the h biases, each value of 
was run 100 times (for each coupler), with each run con¬ 
sisting of 1,000 annealing cycles. The median probabil¬ 
ity Pij{ tt V II taken over the 100 

calculated from the runs, from which we calculate the 
“median” = o:{pij{ tt V || For each 

coupler, a line was fit to {}, yielding a slope 


Pij = l/Tij and an intercept As for /i, we define the 
mean temperature ^ where m is the 

number of couplers. 

Figure l^a) shows the median quantity for evenly 
spaced values of in [-0.1, 0.1], as well as a line fit 
thereto. The closeness of the fit of the line confirms the 
accuracy of the thermal model as for 

Define to be the estimate for 

O f'J f'J 

using the data from the k-th iteration, with = 0 by 

convention. In the kth iteration, we set the programmed 
values of the couplers to 

ty = - E (12) 

k' = l 

i.e. by subtracting the sums of the residual biases from 
the prior iterations from the desired values. Figures [^d- 
f) show the narrowing of the distribution of residual J 
biases with correction. 
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FIG. 3. J biases in the Burnaby device. Data from a series of three experiments (A; = 1, 2, 3) in which for each experiment 
the sums of biases estimated in the previous ones are subtracted from the original programmed values, using Eq. The 
hrst experiment is without any correction, and the second and third use increasingly accurate corrections. All quantities are 
calculated using the mean of the qubit temperatures, calculated indepently in each experiment, (a) From only the first 


experiment, the median quantity over the couplers, of the quantities | for evenly spaced values of 

in [-0.1,0.!]. (b) The standard deviation over the couplers of the estimated at each value of the original (c) For 

a single, typical coupler, (41, 47), the standard deviation over 100 runs of the estimates |e/ 4 ij* 47 | of J 4 i ,47 versus the original 


(d-f) Residual biases 


estimated from each of the experiments. 


Unlike the case for the h biases, for which data indicate 
that the distribution is essentially converged after a sin¬ 
gle iteration, here we see two new phenomena. First, the 
distribution continues to narrow between the second and 
third iterations. Second, the distribution of the resid¬ 
ual biases from the second iteration, while narrower than 
that from the first, is not centered around zero. We be¬ 
lieve this is due to overcorrection; that is, the estimates 
of the biases from the first iteration have a high degree of 
uncertainty, and so simply subtracting their values from 
the intended value introduces some amount of bias itself. 
This is consistent with the overall small magnitudes of 
the J biases relative to those of the h biases, especially 
as compared to the corresponding noise levels. This over¬ 


correction can be mitigated by weighting the correction 
in a way that accounts for the uncertainty in the estimate 
using Bayesian reasoning. 

Figurel^b), analogous to Fig.l^b), shows the standard 
deviation crj over the qubits of the estimates \ Jij \ of 

{Jij^ for different values of for each of the three 
iterations of experiments as described above. The es¬ 
timates were calculated independently for each 

iteration k using its mean temperature There is a 

significant change from the first iteration of corrections, 
but then the standard deviation remains approximately 
the same after the second iteration. Considering the av¬ 
erage over the values of , the overall variance is about 
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the same before and after correction, yet is much more 
uniform after correction, which we consider beneficial. 

Figure [^c), analogous to Fig. [^c), shows the stan¬ 
dard deviation (Jj{r,k) of the estimates {^ 4147 ) = 

'^ 41,47 ^ ^ 

The correction seems to have no effect, 
and the there is a consistent increase in the variance with 
increasing magnitude of the programmed value A 

similar phenomenon occurred in the analogous h data. 


IV. EFFECT OF CORRECTION ON 
PERFORMANCE 

Ultimately, the goal of calibration is to optimize the 
performance of a quantum annealer on problems of com¬ 
putational interest. It is not clear a priori that the biases 
present in one- and two-qubit experiments are the same 
as those present in anneals involving hundreds of qubits. 
Even if they were, their estimation would be of no prac¬ 
tical value unless their correction improves performance. 
To address this, we tested the effect of correcting the h 
biases on the performance of the quantum annealer at 
NASA Ames, using the same parameterized random en¬ 
semble of instances used in a previous study benchmark¬ 
ing a D-Wave quantum annealer [9] . To generate a single 
instance with range r, the local fields {hi} were uniformly 
set to zero, and each available Jij was independently and 
uniformly selected from {—r, —r + 1 ,..., — 1 , 0 , 1 ,..., r}. 
The resulting instance was then scaled by the overall fac¬ 
tor 0.9/r so that the largest magnitude \Jij\ was 0.9. 
(This was necessary, rather than scaling to 1 as in previ¬ 
ous studies, to allow for consistency with future experi¬ 
ments in which the J biases are corrected.) 100 such in¬ 
stances were generated and run twice with 1,000 anneal¬ 
ing cycles for each of the same (uniformly randomly gen¬ 
erated) 10 gauges. In all runs, {Jij} were programmed 
as in the instances. For the first set of runs, which we 
call “uncorrected”, the {hi} were also programmed as in 
the instances, i.e. to zero. For the other set, which we 
call “h-corrected”, the local fields were programmed to 
the inverse of the biases computed via experiments as in 
Eq.[TT] 

For each instance, the uncorrected and corrected re¬ 
sults were compared using two methods, a “greedy” one 
and the elite mean. The greedy comparison is as follows: 
the energies of all states returned were computed, and 
those for all gauges were grouped together. Whichever 
method (uncorrected or corrected) returned the lower 
minimum energy was deemed to have performed better. 
If the minimum energies were the same, the tie was bro¬ 
ken by the number of times that energy was returned. If 
this number was the same, the method with the second- 
lowest energy was deemed to have performed better, with 
ties broken by the number of times the second-lowest en¬ 
ergy was returned, and so on. 

The “elite mean” score function [21] , a quantity previ¬ 
ously introduced to allow comparison of the performance 


of different programming parameters in quantum anneal¬ 
ers when the success probabilities are too low (and thus 
noisy), is defined as the mean energy of the “elite” states, 
i.e. those with the lowest energies. The elite mean is pa¬ 
rameterized by the fraction of energies over which to take 
the mean; here we use 2%. The results of the comparison 
are summarized in Table |Tj showing the proportion of in¬ 
stances, for each r, for which the correction improved the 
performance, using each of the two comparison methods 
described above. For r = 1 and r = 2, there were 6 and 
2 instances, respectively, for which the elite mean com¬ 
parison was tied, all but one due to success probabilities 
greater than 2 % for both the corrected and uncorrected 
experiments. The data set is too small to imply definite 
conclusions, but does indicate that correcting for the h 
biases, even using data from one experiment, improves 
performance according to reasonable metrics. 

TABLE I. Comparison of performance with and with 
/i-correction on benchmarks. 

The probability that correcting for h biases (using data from 
a single experiment) improved performance on 100 random 
instances from an ensemble parameterized by the range of 
values r. Performance was compared according to two 
metrics: greedy comparison of the energies and degeneracies. 


and comparison of the elite mean score function. 


Range rj 

1 

2 

4 

8 

16 

Greedy 

0.58 

0.63 

0.59 

0.68 

0.53 

Elite mean 

0.65 

0.65 

0.73 

0.72 

0.67 


V. CONCLUSIONS 

Imprecision in the programmable parameters of a 
quantum annealer can significantly degrade its perfor¬ 
mance. Some amount of imprecision is unavoidable in an 
analog device, but we have shown how to correct that 
part of the deviation from the desired parameters that 
is persistent and systematic. In this work, we present a 
methodology for determining, in parallel, the persistent, 
systematic biases in all of the available programmable 
parameters {hi, Jij} of quantum annealer that requires 
a relatively small amount of total annealing time and 
whose results remain valid for weeks on current devices. 
The strategy presented here exploits the finite temper¬ 
ature of quantum annealers by using values of the pro¬ 
grammable parameters such that hi, Jij < kT', in this 
regime, the results of one- and two-qubit experiments 
are well described by a Boltzmann distribution, from a 
model of which the biases of interest can be extracted. 

We applied our method to two D-Wave devices, one lo¬ 
cated at NASA Ames Research Center in Moffett Eield, 
California, and one located at D-Wave Systems in Burn¬ 
aby, Canada. The application of the method to both 
devices lead to more uniform statistics of one- and two- 
qubit experiments. In a small test of standard benchmark 













instances using all of the available qubits and couplers 
of the NASA device, the method improved performance. 
Further studies on both the devices used here, as well as 
the only other available device (“LM-USC”), located at 
the University of Southern California-Lockheed Martin 
Quantum Computation Center in Marina del Rey, Cali¬ 
fornia are currently in progress. 

Data collection was significantly impaired by recurrent 
maintenance on the NASA device and the recency of our 
access to the Burnaby and LM-USC device. More com¬ 
plete data from all three devices, currently being col¬ 
lected, will provide a stronger characterization of the 
noise, especially the effect of the programmed values of 
neighboring fields {h!f\ oii fhe effective value of a 

given hi or Jij , as well as enable the analysis of extensions 
of the methods. 


The estimates of the biases can likely be made more 
accurate, as judged by their effect of their correction on 
subsequent estimations thereof and on performance on 
benchmark instances, given the same amount of anneal¬ 
ing time. First, preliminary data indicates that calculat¬ 
ing the J biases without the assumption of no h biases 
(i.e. using Eq. 10 as opposed to Eq. refeq:approx-alpha- 
ij) leads to more consistent estimates. Second, disentan¬ 
gling the mutual effect of the h and J biases on each 
other by alternating between iterations of the iterations 
of h and J experiments (as opposed to doing each alone 
as reported) will likely lead to more accurate estimates of 
each individually. Lastly, the risk of overcorrection can 
be mitigated by weighting the correction by the degree 
of certainty of the estimate of the bias to be corrected. 

Although we focused initially on a standard random 
ensemble of Ising instances for benchmarking the per¬ 
formance of quantum annealers, the effect of correct¬ 
ing biases should be greatest on instances whose ground 
states are most sensitive to misspecification of the pro¬ 
grammable parameters. On average, this sensitivity in¬ 
creases with the range r of the random ensemble, because 
the values of the parameters are scaled to fit within the 
finite physical range while the biases remain at the same 
absolute scale. At a large enough range r, however, even 
correction of the biases is not enough, and inherent fluc¬ 
tuations lead to almost zero success probabilities. This 
would explain the possible pattern seen in the elite mean 
comparison (Table I|) that the advantage of correction 
peaks seems to peak at the level of r considered to cor¬ 
respond to the precision limit of the device. (That such 
a pattern is not as apparent in the greedy comparison 
is easily explained by natural noisiness of that compari¬ 
son method, especially for instances with extremely low 
success probability as was the case here.) 

There is reason to suspect that correction will also have 
a beneficial effect in reducing the effect of gauge selection 
on success probability. While there are other suspected 
reasons for the effect of gauge selection (which would 
be non-existent in an ideal device), that biases could be 
one factor is plausible. The effect of gauge selection is 
significant, sometimes leading to an orders-of-magnitude 


difference in the success probabilities, and so this is a 
promising avenue for bias correction. 

Importantly, while the J biases determined here are in 
general smaller than the h biases, numerical studies indi¬ 
cate that often instances are more sensitive to misspecifi¬ 
cation in the J parameters than in the h parameters [27]. 

The methods presented here complement a growing 
suite of tools for optimal programming of quantum an¬ 
nealers EUHl], tuning the performance thereof to cope 
with the intrinsic noise in current and future physical 
implementations. 
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Appendix A: Quantum annealing 

1. Computational problem 

QA is designed to solve the classical problem of finding 
the ground state of an Ising system, i.e. finding an as¬ 
signment of a set of n classical spins s = G {±1}’^ 

that minimizes a given energy function. Any such func¬ 
tion of spins has a unique representation as a sum of 
monomials of groups of spins, and generally the order 
(locality) of such monomials is restricted to some con¬ 
stant. For any constant locality, a polynomial number of 
ancillary qubits can be introduced and a quadratic (2- 
local) energy function defined on the original and ancilla 
qubits such that the ground state of the original energy 
function can be directly inferred from the ground state of 
the 2-local energy function. For that reason, we restrict 
our attention to only 2-local energy functions of the form 

E{s) = y] hiSi + y] JijSiSj. (Al) 

l<i<n 

The Ising problem is completely equivalent to the 
Polynomial Unconstrained Binary Optimization 
(PUBO) problem, defined as the minimization of 
pseudo-Boolean functions, i.e. real-valued polynomials of 
{0, l}-valued bits, through a linear relation between spins 
and bits. The restriction of PUBO to 2-local function is 
known as QUADRATIC Unconstrained Binary Opti¬ 
mization (QUBO). Much of the literature on mapping 
arbitrary problems to quantum annealers and reducing 
the locality of energy functions uses the PUBO formal¬ 
ism, but we refer here only to the Ising formalism because 
it is more closely related to the physical implementation. 
The Ising problem is NP-hard [24] . 

In practice, physical constraints on the location of 
qubits and their couplers disallow such arbitrary connec¬ 
tivity, with an interaction between any pair of qubits. 
A further step known as embedding allows for hardware 
with restricted connectivity to solve problems of the form 
in Eq. |A1| by mapping each logical qubit to a connected 
set of physical qubits and introducing strong ferromag¬ 
netic couplings between them to make them act as one. 

2. Physical process 

Quantum Annealing (QA) is a metaheuristic for solv¬ 
ing combinatorial optimization problems [28]. The main 
procedure is this: a final Hamiltonian is constructed for 
some set of qubits such that its ground state encodes 
the optimal solution to the desired problem, those qubits 
are prepared in the ground state of an initial Hamilto¬ 
nian, and then over the course of the annealing process 


the initial Hamiltonian is continuously transformed into 
the final Hamiltonian. While there is ambiguity regard¬ 
ing the exact definition of the term “QA” and its dif¬ 
ferences with its closely related spin-off, adiabatic quan¬ 
tum computation [29], we define it here to allow for non- 
adiabiticity in the annealing process and finite tempera¬ 
ture, on the latter of which our methods are based. 

To solve a computational problem, a Hamiltonian is 
constructed whose ground state encodes the optimal so¬ 
lution of the given problem, 

E E (A2) 

l<i<n 

i.e. the quantum analog of The system is ini¬ 

tialized in the easily-prepared ground state of another 
Hamiltonian, usually 

E A3) 

l<i<n 

whose ground state is a uniform superposition of all 2^ 
computational basis states. Then the Hamiltonian is 
slowly changed from the former to the latter; explicitly, 

H{t) = + 5(t)i/final, (A4) 

where A{t)^B{t) > 0 define the “annealing profile” and 
are such that A(0), H(tanneai) >> 0 and A{T) = B{0) = 
0, where tanneai IS the annealing time. 


3. D-Wave devices 

D-Wave devices consist of an array of coupled super¬ 
conducting flux qubits that is effectively a Ising spin 
system with programmable spin-spin couplings and lo¬ 
cal fields, longitudinal and transverse. The qubits are 
arranged in a so-called Chimera topology consisting of 
a square lattice array of bipartite unit cells. The two 
devices used in this work consist of 512 nominal qubits 
in an 8-by-8 array of 8-qubit unit cells. Post-fabrication 
testing indicated that only 509 and 424 qubits were us¬ 
able in the NASA and D-Wave devices, respectively. In 
both devices, the programmed values of {hi} and {Jij} 
are specified by unitless parameters in [—2, 2] and [—1,1], 
respectively, where 1.0 corresponds to an actual energy of 
3.2 GHz. Further details of the hardware can be found 
in [30]. Thoughout this paper, we use the standard 1- 
based indexing for the qubits of a D-Wave device with 
512 qubits, as in, e.g.. Fig. 7(b) of [11]. (While the set 
of functional qubits differs between actual devices, the 
convention is to still assign the indices to all qubits and 
to disregard the broken ones.) 
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Appendix B: Persistency of the biases 


Fig. 


shows the correlation of the biases as estimated from single experiments repeated at different times. 


Each data point corr espond s to the bias in each of the 509 qubit of the NASA device. All experiments were performed 

with 100 runs of 1000 annealing cycles each for 41 evenly spaced values of in [- 

The experimentally determined median 


III A 


as described in Sec. 

0.1,0.!]. Notice the strong correlation for intervals greater than one month 


temperature can also be considered constant, with values within 1% from the average of these four realizations: 

= 19.0,19.2,19.2, and 19.2 mK, for the experiments on 10-09-2014 at 14:00, on 10-09-2014 at 16:30, on 10-24-2014 
at 11:15, and on 11-12-2014 at 7:45, respectively. 


(a) Same day (b) 15 days apart (b) 34 days apart 



FIG. 4. Persistency of systematic biases for time intervals. 


Appendix C: Qubit temperatures 


Figure [^a) shows the distribution of estimated “qubit temperatures” {Ti}. Recall that each qubit temperature Ti 
extracted a linear fits of the points e.g. Fig. |^c). That the variance of the temperatures of the qubits 

is as large as indicated here seems unlikel y. For that reason, a more physically plausible “device temperature” was 
estimated, i.e. T or T, as described in Sec. 


Ill A 


Data indicate that the variance is partly systematic, i.e. not due to 
the fluctuation of the experimental values on which the estimates is based. We suspsect that susceptibility [31] and 
cross-talk may play a role here. 

Figure l^b) shows, for three related experiments on the NASA device, the standard deviation a~ over the qubits 


of the estimated value for different values of One experiment was without any correction; the other two 
used corrections calculated in two different ways, one using the qubit temperatures {Tj} and the other the median 
temperature Both corrections clearly improve the variance, but the similarity of the degree of improvement 

justifies our use of an effective device temperature. 



FIG. 5. Qubit temperatures and their effect, (a) The estimates of the individual qubit temperatures {Ti} calculated 
from a single experiment without correction, (b) The standard deviation cr= of the estimated field values hi for different values 
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